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Abstract 
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insights into cognitive function. Secondly, it is used to attenuate eye-related artifacts in 
electroencephalography (EEG). Thirdly, EOG serves as a helpful means for calibrating covert 
brain activity with overt behavior. We provide an overview of the first two application areas, and 
we illustrate the third application using an example data set capturing an "aha moment" in our 
research in the area of mathematical problem solving. 
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Introduction 

Many educational researchers are concerned with understanding how teachers and learners are 
thinking. That is to say, many educational researchers are concerned with understanding what is 
going on in the minds of teachers and students to better assess what they know, how they know 
it, and thereby to help design pedagogies that can lead to improvements in helping these 
populations to be educated more effectively. Traditionally, educational researchers engaged in 
the empirical study of cognition and learning have relied upon overt behavioral data gathered 
from interviews, field notes, self-reports, and audiovisual recordings. In focusing on overt 
behavior, educational researchers have typically not been privy to what is happening covertly, 
with regard to the physiology of these populations. To the extent that cognition and learning are 
embodied behaviors, it would be helpful to augment these traditional data sets with physiological 
data sets whenever possible and to the extent that it is practical to acquire them. We know from 
cognitive neuroscience and psychophysiology, that there is much to be gained from informing 
educational research with results from these well-established fields. Educational neuroscience is 
attempting to accomplish just that. As educational neuroscientists, one challenge we have 
encountered in so doing concerns how best to connect our traditional overt data sets with the 
covert behavioral data sets, specifically, in our case, electroencephalography (EEG), to obtain 
deeper educational insights into cognition and learning. 

One way of investigating connections between these overt audiovisual (AV) data sets 
with covert physiological recordings is through eye-related behavior (ERB), which can be 
observed using electrooculography (EOG) and eye-tracking (ET) technology. In this paper, we 
focus on the potential roles and importance of EOG for mathematics educational neuroscience 
research. Electrooculography enables accurate measurements of ERB (i.e., blinks & movements) 
by recording changes in voltage potentials generated as a consequence of ERB. We identify and 
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discuss three main uses of EOG. First, EOG provides insights into cognitive function. Secondly, 
it is used to attenuate eye-related artifacts in EEG. Thirdly, EOG serves as a helpful means for 
calibrating covert brain activity with overt behavior. In this paper, we present some background 
regarding educational studies concerning ERB using ET and describe EOG. We then provide an 
overview of the first two of the three aforementioned application areas for EOG. We illustrate 
the third application area using an example data set capturing an "aha moment" obtained in the 
course of our research in the area of mathematical problem solving. 

Background 

A major reason for growing interest in educational neuroscience in mathematics education 
research, aside from the now widespread recognition that cognitive constructs are embodied, is a 
need for better empirical grounds for developing theories of mental functions and processes 
(Campbell, 2006 a, b). This, in turn, can reconstitute the conceptual basis for more effective 
forms of mathematical learning and instruction. According to Byrnes (2001), brain research is 
relevant to the field of psychology and education to the extent that it fosters better 
understandings of mind, development and learning. The validity, reliability, and relevance of 
psychological theories of teaching and learning developed from traditional psychological 
experiments may variously be corroborated, refined, or refuted through neuroscientific studies or 
the use of neuroscientific tools and methods to test hypothesizes of any particular theoretical 
account (cf. Byrnes, 2001; Kosslyn & Koening, 1992). 

Conversely, research in the neurosciences can also benefit from the more situated and 
ecologically grounded insights into cognition and learning that typifies the concerns and aims of 
educational research. Accordingly, educational research that informs and is informed by 
neuroscientific research, while incorporating methods of cognitive neuroscience with the further 
aim toward corroborating, refining, or refuting certain models of cognition and learning — that is 
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to say, educational neuroscience — should become of abiding interest and concern to educators 
and educational researchers alike. This interest includes, beyond standard methods of data 
collection in educational research, analyses of EEG data sets measuring bioelectrical signals 
generated from brain activities engaging cognitive processes, such as those involved in 
mathematical thinking. 

Theoretical considerations concerning brain dynamics and embodied cognition aside, 
there are also methodological challenges in isolating brain activity with EEG and in calibrating 
EEG with other eye-related behavioral data sets, such as EOG and ET, along with traditional 
behavioral data sets, such as AV recordings. As our main objective here is methodological, we 
turn now to ways in which EOG can help address these matters. 

Electrooculography 

In the middle of nineteenth century Emil du Bois-Reymond observed that the cornea of the eye is 
electrically positive relative to the back of the eye, forming a so-called comeoretinal potential in 
the range of 0.4 - 1.0 mV (Malmivuo & Plonsey, 1995). Since this potential was not affected 
sufficiently by light or any other environmental conditions, it was considered as a resting 
potential. As the eyes move, each of them behave like single dipoles oriented from the retina to 
the cornea. Eye movements produced these moving (rotating) dipole sources and signals from 
these sources are measured by EOG in the range of 5-20pV/° (see Figure 1, below). Data are 
acquired by electrodes placed around the eyes; these electrodes detect the rotations of the 
electrostatic dipoles. Figure 1 illustrates the kinds of signals generated from horizontal eye 
movements, as recorded by a bipolar pair of electrodes attached to the sides of the left and right 
eyes, lateral to the external canthi (just laterally and external to the left and right of each eye). 
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Figure 1 : An illustration of the electro-oculogram (EOG) signal generated by horizontal 
movement of the eyes. The polarity of the signal is positive at the electrode to which the 
eye is moving (from Malmivuo & Plonsey, 1995). 



When the eyes are at rest, the electrodes detect resting potential, so no voltage is 
recorded. As illustrated in Figure 1, a rotation of the eyes to the right result in a difference of 
potential, with the electrode in the direction of the movement becoming positive relative to the 
second electrode. A rotation of the eyes to the left results in the opposite effect. As with most 
techniques, EOG has both advantages and disadvantages. The main limitation of the method is 
the difficulties owing to artifacts arising from extraocular muscle activity (see below). The 
advantages of this technique include minimal interference with eye movements. 

Electrooculography as a Cognitive Indicator 

It has long been said that the eyes are the windows of the soul. Poetics aside, insofar as the 
soul is an antiquated term for the mind, it seems reasonable to consider that ERB could 
provide some indication of various cognitive states, functions, and behavior as well. For 
instance, ERB, particularly eye-movements, have been studied using ET as an indicator of 
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attention and memory (Kramer & McCarley, 2003), and pupillary response has been studied 
as an indication of cognitive load (Just, Carpenter, & Miyake, 2003). 

Many kinds of mathematical activity are connected with ERB, which can variously be 
voluntary or involuntary. Reading (including reading mathematical texts) serves as an example 
of voluntary eye movements and eye movements can be predictable to some extent. Less 
deliberate to involuntary eye-related behavior are eye-blinking and saccades (rapid macro and 
micro eye-movements). Behaviorally, the specific character of eye movement during reading is 
summarized in Duchowski (2002). His study is based on ET research and its application. 
Duchowski hypothesizes and cites evidence that as the text becomes conceptually more difficult, 
fixation durations increase and saccade lengths decrease. Eye movement and eye fixation 
analyses have also been used as a method of research into strategies of successful and 
unsuccessful arithmetic word problem solvers (Hegarty, Mayer, and Monk, 1995). According to 
Hegarty, et al's study, less successful problem solvers fixate their eyes on numbers and relational 
terms when they re-read parts of arithmetic word problem, whereas more successful problem 
solvers fixate eyes on variable names. They propose that successful problem solvers construct 
problem models and concentrate their attention on appropriate variable names, whereas 
unsuccessful problem solvers attempt to directly translate key propositions of these problems 
into a computational procedure, and thereby remain predominantly focused on numbers. 

More generally, studies using EOG provide evidence that blinks and saccades are 
indicators of cognitive function. For instance, it has long been known that blink rates decrease 
significantly during visually demanding tasks (e.g., Fogarty & Stem, 1988). Moreover, blinks 
and saccades evidently serve to punctuate shifting foci of attention, be they exogenous shifts in 
attention from one object to another, or endogenous shifts in reflection from one line of thought 
to another (Bonfiglio, Sello, Andre, Carboncini, Arrighi, & Rossi, 2009). 
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Electrooculography for Artifact Correction 

All eye-related behavior (viz., ERB), i.e., eye-movements, eye-blinks, saccades, dilations, and 
focusing, voluntary or not, are based on extraocular muscles encasing the eyes (see Figure 2), 
though we should note that other muscles associated with the neck, scalp and face are also 
implicated with ERB, when, for instance, one is moving one’s head and/or wincing. 




Figure 2: Extraocular muscles: 1) the Levator Palpebrae Superioris muscle elevates the 
eyelid; 2) the Superior Rectus muscle makes the eye look upwards or medially or wheel- 
rotates it medially (intorts); 3) the Superior Oblique moves the eye downwards or 
laterally or wheel-rotates it inwards (i.e. makes twelve o’clock on the cornea move 
towards the nose); 4) the Medial Rectus is the largest of the ocular muscles and stronger 
than the lateral and is a pure adductor; 5) the Lateral Rectus makes the eye look directly 
laterally in the horizontal plane; 6) the Inferior Oblique makes the eye look upwards or 
laterally or wheel-rotates it; 7) finally, the Inferior Rectus makes the eye look downwards 
or medially or wheel-rotates it laterally. This muscle also serves to depress the lower lid. 
Other ocular muscles (not shown here) actually enclose the eye. 



Voltage potentials from motor neurons of extraocular muscles causing muscle 
contractions interfere with both EOG and EEG, the latter of which monitors brain activity. We 
shall see that EOG enables improved correction of these interfering potentials in EEG data, 
thereby improving the quality of EEG data and subsequent EEG analysis and interpretation. 
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EOG data also enable time synchronization and integration of EEG with ET and AV data sets by 
calibrating EOG data with these data sets. 

According to Kierkels, Boxtel, and Vogten (2006), before brain activity measured by 
EEG through scalp voltage potentials is analysed — especially, we would add, in single trial 
(non-replicable) conditions — ERB artifacts should be identified and attenuated using signal 
processing techniques. Figure 3 illustrates ERB artifacts in EEG data. 




Figure 3: This raw data set illustrates strong eye-related artifacts, evident in the EOG 
channels in the bottom of the figure, in the rightmost two thirds of the figure in the frontal 
EEG channels, which are closer to the eyes (in this case, the upper and middle channels). 

It is helpful to mention here that when analyzing EEG data, the signals generated by 
ERB, as well as any other voltage potential signals of non-cerebral origin, should be treated as 
artifacts (i.e., undesired signals). Puthusserypady (2005) notes that ERB signals are typically the 
most significant and common artifacts in EEG, and that removal of ERB signals from EEG data 
forms an important part of preprocessing of EEG data prior to EEG analysis. Eye blink signals 
are of high amplitude. Indeed, they can be as much as ten times the amplitude of signals due to 
brain activity recorded from the frontal EEG channels, and they are low frequency in nature and 
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affect the low frequency range of EEG signals. Kierkels et al (2006) point out that ERB signals 
have large disturbing effects on EEG recordings because the eyes are located so close to the 
brain. 

There are many methods for eye movement correction in EEG recordings reported in 
literature, with varying degrees of success (Kierkels, et al, 2006; Croft, Chandler, Barry, Cooper, 
& Clark, 2005). Ille, Berg, and Scherg (2002), for instance, distinguish between methods that 
remove ERB signals from EEG data without considering brain activity and those that do attempt 
to separate artifacts and brain activity. In the former case, in multi-trial studies, signals that 
exceed a threshold criteria, such as maximum amplitude, beyond which are typical of eye-blinks, 
are rejected whole scale, together with the EEG data. In the latter case, these authors also 
consider signal processing methods, such as Principle Component Analysis (PCA), Independent 
Component Analysis (ICA), Multiple Source Eye Correction (MSEC), that are designed to 
separate ERB artifacts and EEG brain activity, as promising methods for future development. In 
our presentation (see Appendix I), we demonstrate the ICA method for separation ERB signals 
from brain activity signals by subtracting components related to ERB from EEG data. 

Electrooculography for Calibrating Data Sets 

We use EOG recordings in tandem with ET and AV data sets to synchronize with EEG data sets, 
which are indirect recordings of the electrical component of the biopotential field recorded on the 
scalp being generated from brain activity. We illustrate our approach with a data set capturing an 
“aha moment” during a mathematical problem solving task. It should be mentioned here that the 
mathematical problem solving experiments are principally of so-called ‘single trial’ type as they 
cannot be fully replicated or repeated, either with regard to stimuli or thought processes. 
Every such experiment is a unique combination of cognitive process that requires observations of 
high signal fidelity and integrity for analysis. In integrating overt and covert behaviors associated 
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with problem solving events, we aim to explore the extent to which valid and reliable 
information can be obtained from single trial studies in mathematical problem solving. 

In the single trial data set we present, the mathematical problem task was based on the 
paradigm of Dehaene, Izard, Pica, & Spelke (2006). These data involve a slide comprised of six 
diagrams of different kinds of quadrangles: Five of the six diagrams were connected by a 
common mathematical concept of ‘diagonals of the quadrangles’. Note that the dots inside of 
five of the six diagrams were located exactly at the intersection the diagonals; the task was to 
identify the diagram that did not conform, that is, the diagram where the dot inside the 
quadrangle is not located at that intersection. This paradigm was presented on the screen of a 
computer to one male participant, aged 38 years and 3 months. 

Although we are certainly interested in removing ERB signals from EEG data (again, see 
Appendix I), we are also interested in using EOG data to synchronize brain activities manifest in 
EEG with our eye tracking data and with our audiovisual behavioral data (see Figure 4). 




(a) (b) 



Figure 4: An integrated time-synchronized data set. Notice that EOG channels 4 and 7 
(from the top in figure 4a) express the dipole characteristics schematized in Figure lb. 

Figure 5 is a frame from a movie file demonstrating the process of thinking on this problem in 
terms of EEG and EOG signals. We demonstrate the utility of EOG data to help accurately 
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calibrate, i.e., time-synchronize these brain data with the behavioral data. 




(a) (b) 



Figure 5: Fragment of movie demonstrating the process of mathematical thinking (and 
“aha moment”) in terms of EEG, FOG and respiration signals (a); detailed segmentation 
of EEG data on the basis of FOG waveform analysis (b). 

Figure 6 provides a detailed breakdown of 10 saccades, beginning with the initial eye 
movement "1" terminating at "dO", which is the text region in the upper right comer of the slide, 
followed by eye movement "2" down to diagram "dl" and so on. The expansion of the ancillary 
channels in Figure 5b shows data obtained from the FOG electrodes, attached just off the comers 
of the participant's left and right eyes, highlighting "gaze" regions (dO, dl, ..., d6), and eye 
"move(ment)" events (1, 2, 3, ..., 10). The annotations in Figure 5b correspond to the annotations 
indicated in Figure 6b. The shaded areas in Figure 5b denoted by the gaze regions indicate when 
the participant was attending to the six diagrams in the slide in Figure 6a, with "dO" indicating 
the text region to the upper left. These annotations are most clearly indicated in Figure 6b. 

(a) (b) 

Figure 6: Actual eye-movements on the screen stimulus, following Dehaene, Izard, 

Pica, & Spelke (2006) (a), and a schematization of those movements (b). 



Geometry: Diagonals 
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Getting accurate calibrations are crucial to properly associating covert brain activities 
with overt problem solving tasks. Once these data sets are synchronized, a more integrated 
approach can be taken in connecting brain and behavior. Such integrated approach serves as 
method for validation and reliability of mathematical problem solving data interpretation. Our 
interpretation of these data is an entire topic of its own, and as such, is left for another time. 

Conclusion 

Eye-related behavior can be used to infer cognitive function and to operationalize human 
performance. From a cognitive perspective, it can be variously considered as sensory-driven, 
emotionally-responsive, or goal-directed. In terms of performance, eye-related behavioral 
indexes, like fixation patterns and gaze durations, can operationalize efficiency and workload. 
Hence there are implications for both studies in cognition and learning, and instructional design 
and usability of educational software. Eye related behavior manifest in EOG signals thereby 
serve as indicators of various aspects of cognition and performance, including aspects of 
attention and memory evident in mathematical problem solving. Eye-related behavior evident in 
EOG can be both a blessing and a curse, however, insofar as ERB activity interferes with EEG 
recordings that may provide further insights and indicators in this regard. Fortunately, EOG 
signals can be used with signal processing methods such as ICA to help attenuate those artifacts 
from EEG data. Finally, it is evident that EOG data can serve as a bridge in connecting EEG and 
other physiological data sets with more overt behavioral data sets such as eye-tracking and 
audiovisual data sets. Indeed, EOG serves well in a variety of ways in connecting brain and 
behavior, thereby increasing the validity and reliability of single trail studies. We hope to have 
demonstrated that EOG has important methodological roles to play in helping to establish 
educational neuroscience as an important and viable new area of educational research. 
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Figure A1 : Raw EEG data. 
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Figure A2: Raw EEG data with vertical (A3) and horizontal (A4) eye movements removed. 
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Electrooculography. . . 
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